AI misleading content: who’s responsible?
Organisational Epistemic Responsibility in the Age of AI: Research Insights
In my 2024 Master's dissertation in Philosophy & AI at Northeastern University, London, I explored the critical intersection of organisational responsibility and AI-generated content. My research revealed how the rise of artificial intelligence, particularly Large Language Models (LLMs), presents organisations with both opportunities and challenges. This analysis comes at an interesting time as organisations worldwide grapple with implementing Generative AI systems while maintaining information integrity.
The Evolution of Epistemic Responsibility
My dissertation proposed that epistemic responsibility extends far beyond mere information accuracy. It encompasses an organisation's fundamental obligation to ensure knowledge creation, sharing, and application meet rigorous standards of truthfulness and reliability. In my analysis, I observed how this responsibility has taken on new dimensions with the advent of AI-generated content.
The philosophical foundation of epistemic responsibility rests on three key pillars:
Truth-seeking behaviour: The active pursuit of accurate and reliable information
Knowledge stewardship: The responsible management and distribution of information
Epistemic virtue: The cultivation of intellectual habits that promote accurate knowledge formation
My paper highlighted that modern organisations must approach information verification systematically, manage knowledge distribution carefully, and develop robust processes ensuring information integrity at every level.
Technical Foundations and Challenges
In examining LLMs, I focused on how they operate through sophisticated neural networks using attenuation mechanisms to process and generate text. My research presented specific challenges (features) in their technical architecture that organisations must address. LLMs operate through transformer architectures that process text using attention mechanisms and neural networks. These systems employ self-attenuation to weigh the importance of different words in context, creating probabilistic models of language. Thus, current LLM implementations face significant constraints in maintaining context over long passages, often struggling with comprehensive understanding. Through my analysis, I identified how their attention mechanisms frequently fail to capture crucial long-range relationships in text, while embedding space distortions can affect semantic accuracy in subtle but important ways.
For example, we encounter the problem of inaccurate information generation, often stemming from incomplete or biased training data. Consider, for instance, how an LLM trained predominantly on male-dominated professional fields might perpetuate gender biases in its outputs. Furthermore, these systems frequently struggle with contextual understanding. While their responses might appear coherent on the surface, they can harbour dangerous flaws. A concerning example could involve an LLM trained on medical data recommending aspirin for an infant's fever. This potentially dangerous suggestion might arise from the LLM learning that aspirin is a common treatment for fever in adults, without understanding the critical age-related contraindications
My paper proposes that the LLM model training process itself creates inherent vulnerabilities that organisations must navigate. I documented how information boundaries arising from training data cutoffs limit the model's knowledge scope, while systematic biases from training data distributions can perpetuate existing prejudices. In my assessment, pre-training artefacts can affect output quality in unexpected ways (see my footnote for a classic and real example!).
Risks and Harms
My research identified risks that organisations face when implementing generative AI systems. I observed how operational integrity can be compromised when AI systems introduce incorrect information into critical processes. In my analysis of reputational implications, I documented how brand value erosion can occur rapidly when AI systems generate incorrect or inappropriate content, leading to degraded stakeholder trust. This often attracts increased regulatory scrutiny and can create vulnerabilities in an organisation's competitive position. Furthermore, the societal implications of AI-generated misinformation are far-reaching. By amplifying existing biases and accelerating "truth decay," AI can erode public trust in information and expertise. AI-generated content may inadvertently reveal or misuse personal information, raising serious privacy concerns.
Real-World Manifestations
I uncovered numerous incidents that illuminated these challenges. I studied how Air Canada's chatbot provided incorrect bereavement fare information, leading to legal challenges. I analysed the Australian Senate committee's acceptance of AI-generated misinformation about KPMG, which demonstrated to me how AI-generated content can infiltrate even the highest levels of governance.
My investigation included the World Health Organisation's experience when its virtual health worker distributed outdated medical guidance, potentially affecting public health decisions. I also examined DPD's experience with a chatbot generating inappropriate responses, which further illustrated the reputational risks.
Framework for Responsible Implementation
Based on my research, I proposed that organisations must adopt a structured framework for responsible AI implementation. This framework should address (at a minimum) four key questions:
Is an LLM truly indispensable for the task at hand?
How critical is response accuracy for the specific application?
What mechanisms exist for verifying chatbot responses?
What procedures are in place for addressing and learning from mistakes?
Conclusion
My dissertation discussed that the use of Generative AI systems demands a thoughtful approach to epistemic responsibility. I found that organisations must develop comprehensive frameworks addressing both technical and ethical considerations, supported by robust governance structures and control systems. Success requires not just technological sophistication, but a fundamental commitment to information integrity and responsible innovation.
This analysis draws from my 2024 Master's dissertation in Philosophy & AI at Northeastern University London. My full dissertation provides detailed examination of epistemic responsibility in organisational AI implementation.
As a footnote, I would also like to add that I passed this blog post through Claude.ai which told me that I had studied at the University of Oxford. This is a classic example of a hallucination that was likely generated by the model's statistical analysis of the text. Claude, like other large language models, doesn't possess logic or reasoning abilities. Instead, it identifies patterns and relationships within the massive dataset it was trained on.
In this instance, the model likely encountered the words 'masters,' 'philosophy,' and 'London' frequently associated with Oxford University within its training data. Based on this statistical association, it generated the incorrect output. This highlights the importance of understanding that LLMs primarily rely on statistical patterns and do not possess true comprehension or reasoning capabilities.
(This is my personal blog, so the info here might not be perfect and definitely isn't advice)